Environment Design in Human Computation

Human computation is a new research area that takes advantage of human brain power to solve problems that computers cannot efficiently solve yet. In human computation systems, such as Games with A Purpose (GWAP) and crowdsourcing markets, users are given computationally hard tasks, and get rewards by accomplishing the tasks. In this work, we are interested in how to design a human computation system. In particular, we focus on the situation where developers cannot re-design the whole application but can only make limited changes to the systems. We model this situation as an environment design problem [10] with multiple agents. Under the assumption that agents do not interact, we define a m by n decision matrix, whose entry [i, j] represents the utility value of historical decision agent i made in environment j. Our goal is to find the environment for each agent to maximize the system payoff while minimizing the number of entries needed. To achieve this goal, we propose two different approaches, agent type elicitation and collaborative filtering, under different conditions. We also derive the convergence bound on the number of entries needed for each method.